fix: preserve pending_hooks on import + tighten StateValidator JSON extraction#124
Open
Nerdless-ship-it wants to merge 1 commit intoNarcooo:masterfrom
Conversation
BUG 1 — state-validator: extractBalancedJsonObject would accept a candidate
that ended at '}' but had non-structural trailing content (e.g.
'{...} more text here'). tryParseJson would then fail on the full
candidate string, causing 'State validator returned invalid JSON'.
Fix: after finding the matching '}', verify the immediately following
character is nothing, whitespace, or a structural JSON token before
accepting the candidate.
BUG 2 — import replay: resetImportReplayTruthFiles overwrote
pending_hooks.md with an empty seed before chapter replay, wiping out
all existing hooks (35 → 0, then replay generates ~16 from scratch).
Fix: remove the writeFile for pending_hooks.md in resetImportReplayTruthFiles.
Hooks are chapter-content-specific; preserving them during import
avoids data loss. The replay will incrementally add new hooks without
destroying existing ones.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two bugs found while writing a 20-chapter novel with inkos:
BUG 1 — StateValidator JSON extraction accepts trailing content
File: packages/core/src/agents/state-validator.ts
Problem: extractBalancedJsonObject finds the matching closing } but does not validate what comes after it. When the LLM response contains text after the JSON block (e.g. "{...} more text here"), the candidate passed to tryParseJson includes that trailing content, causing JSON.parse to fail with "State validator returned invalid JSON".
This was observed on chapters 23 and 24 in v0.6.3 even after the character-scanning fix replaced the old greedy regex.
Fix: After finding depth === 0 at }, check that the immediately following character is either: nothing, whitespace, or a structural JSON terminator (], }, ,). If any other character follows (e.g. alphabetic text), reject the candidate and continue scanning. This is safe because those characters cannot legally appear immediately after a valid JSON object in the validators output format.
BUG 2 — import chapters destroys pending_hooks.md
File: packages/core/src/pipeline/runner.ts
Problem: resetImportReplayTruthFiles (called at the start of a fresh chapter import) overwrites pending_hooks.md with an empty seed (buildImportReplayHooksSeed). This wipes all existing hooks before the replay begins. After replay finishes, only the hooks detected from the imported chapters remain — typically ~16 instead of the original 35.
For a novel that has been in active development, this is a significant data loss.
Fix: Remove the writeFile for pending_hooks.md in resetImportReplayTruthFiles. Hooks are chapter-content-specific; preserving them during import avoids data loss while still allowing the replay to incrementally add new hooks detected from the imported chapters.